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(54) Multiple path routing 

(57) A novel data structure in a router helps to com- 
pute viable next hops for forwarding a data packet from 
a router to its destination along multiple alternate loop- 
free paths, which are not necessarily of shortest dis- 
tance. Each viable next hop may also be specified with 
a degree of optimaiity, which enables a route to perform 



QoS routing and fault-tolerant routing efficiently. The da- 
ta structure can be implemented as an add-on software 
to existing routing protocols and may be implemented 
in existing networks which use shortest path protocols, 
even where less than all of the routers use the data 
structure and multiple path scheme described herein. 
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Description. 

RELATED APPLICATIONS " t ^ ' ' 

[0001] This application claims the benefit of prpvisibri- 
al U.S. Patent Application Serial Nb: 6O/056,O2> fil^ 
September 2, 19$7. ( "' ... V : ' \ ^ , ! 

HELD OF THE INVENTION r " 

[0002] , Trie present invention relates generally to rout- 
ing data packets to a destination node within a packet 
switching network. In particular the invention relate tp'a 
multiple path routing scheme using a novel data struc- 

ture * ; ; 1/ " " \ ' ' t ' , "V'"? 

BACKGROUND OF THE INVENTION _ t ? 1 

[0003] j In packet switching networks such as the In'r 
ternet, a packet from a source traverses intermediate 
network routers before reaching its destinatipn/'Befpre 
a packet arrives, at a router, a routing protocol is used 
to determine the existence arid status ot all routers j aVid 
links : therebetween on the network: This ihfbirrhation is 
used by the router to determine an apprbprate next hop 
for an arriving packet with a given destination. THe riekt 
hop includes the next router along the path to the aW 
tination and the outgoing port of the present router link- 
ing the next router. A widely used link-state routing pro- 
tocol, is Open Shortest Path First ("OSPF'j 
[0004] Given the current link states in the network, (e 1 
g. ? reduced bandwidth,, increased bandwidth, discon- 
nected, reconnected) each router S keeps track of a 
shortest path tree t "SPT' from S to every other router in 
the network. Since the router retains only one shortest 
path from S for each destination router, the first hop from 
S along this path willbe used as the next hop to forward 
any packets with the same given destination. n [' 
[0005] Although a router using OSPF has global in- 
formation about the regional network topology, it deter- 
mines only the local route of a packet; i.e., the next hop. 
When each router computes such next-hop information 
based on an SPT, each packet is guaranteed to be for- 
warded along a loop-free path to its destination! A loop- 
free path is a path that doesn't include the same router 
morejhari once. Multiple SPTs rooted at router S can 
co-exist in a network. In such a case there are multiple 
choices of next hops from S to forward a packet, each 
of which will jead to a different patfi of the same shortest 
distance. The packet can be forwarded to any of these 
next hops. 

[0006] Bouting protocols allowing for only a single 
shortest path suffer from several inefficiencies. For ex- 
ample, if, immediately before dispatching a packet along 

. its next hop, a router learns that a link along the path is 
down, v the router must, first recompute a , new single 

, .shortest path and only then .send the packet along the 
new next hop. . . „ , , f 



[0007] The single shortest path method also ignores 
network load considerations!^ ''thus, even if a path has a 
lot of traffic the packet will be forwarded along that path 
if it is the shortest to its destination? Furthermore, if is 

5 taown trial findihgari optimal^ath that satisfies Certain 
QdS constraints ' is in ^general [ ^mputatibhally difficult 
(NP-bompjetej. Since some Qbs ^rsjnneters that need 
to be optifnized may depend on the NetWork traffic load 
and thus can change dynamically over a short period of 

1? time, finding an optimal path can become infeastole 
even for a moderate-size network. Even for QbS param- 
eters that do hot change frequently, determining the op- 
timal path from a router may require global knowledge 
of these parameters in the network, and a local change 

\8 in these parameters would therefore need to be dissem- 
inated to every node in the network. Hence in most cas- 
es only suboptimal paths can be computed. 

SUMMARY OF THE INVENTION 

[0008] The present invention uses a multiple path al- 
gorithm (referred to herein as "MPA") to enable a router 
to forward packets to multiple viable next hops that are 
not strictly constrained to be a part of a shortest path 
25 from the router to the packet's destination. The rhethod 
of the present invention still guarantees/ however, that 
all packets will be routed to their destirtatioh on loop-free 
paths. In general each path traversing fa' i different next 
hop is guaranteed to be loop free if it satisfies the con- 
3? straint that the shortest distance from the implementing 
router to the destination decreases at each next hop. 
I00091_Jn a pref erred embodiment of the invention, the 
MPA uses a novel data structure which stores informa- 
tion relating tb each router (node) in the network that is 
35 a potential destination node. In particular, the data struc 1 
ture maintains at least the following attributes: the short- 
est distance from the implementing router to the desti- 
nation node; the cost of each lirikfrom the implementing 
router to each potential next hop; and fbr each potential 
40 next hop, the shortest distance f rom .the implementing 
router to the destination node along a path traversing 
that particular next hop: I ; * : - 

[0010] Using this data structure, multiple viable next 
hops are computed by selecting next hops from the pb- 
46 , tehtial next hops whose shortest distance attribute tra- 
versing that next hop, minus' the ; cost of the link from the 
imp lementing router to the n ext hop , is less than th e 
shortest path from the implementing fbuter to the desti- 
nation node. ■ . 
[0011] The method and data 5 structure 1 of the present 
invention can be efficiently implemented as art add-on 
component F to f existing routing protocols sLjch as OSPF. 
Each MPA implementing router uses the topology infor- 
. mation exchanged bj the router protocols tb compute 
multiple paths between a given source and a given des- 
tination. Moreover, conventional routers based on short- 
r est path routing can co-exist in the same network with 
MPA implementing routers to guarantee loop-free rout- 
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ing for each packet. In general, an MPA implementing 
router can interoperate with any router whose routing 
protocol uses the ; Jopp^free policy. \. \ , , 
[0012] . By maintaining, muftiple viable next hops in a 
router for each destination, recovery frorn link failure is 
much faster, than heretofore possible.. since the recovery 
mechanism, using alternate paths can be implemented 
locally by the router, and.there is almost no. delay in- 
curred by the Jink failure. On the other hand, .. without 
MPA, after detecting a link failure, the router would need 
to broadcast throughout thenetwork the link failure, then 
recompute the shortest path tree to determine a new al- 
ternative path. . , . : 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0013] 

FIG. 1 is a graphical depiction of the data structure 
used in accordance with the present invention. 
. FIG. 2 is a graphical depiction of the data structure 
used in a conventional shortest path tree algorithm. 
FIG. 3 depicts a router architecture employing the 
MPA . algorithrn as used fn accordance with the 
present invention . ls , , , v 

FIG. 4 is,a representation of, a network of routers 
including a source, router S and a destination router 

DETAILED DESCRIPTION OF THE INVENTION , 

[0014] The packet routing method of the present in- 
vention, involves making available at each router, alter- 
native multiple next hops to forward a received packet. 
The only constraint in computing the multiple next hops 
is that the paths to the. packet's destination resulting 
from these next hops dp not return the packet to any 
router it previously hit! In otherwprds, the paths of these 
alternative next hops shoiitd.be loop free. As discussed 
above, the OSRF. protocol well known in the art, guar- 
antees loop-free routing to each destination by con- 
straining each router to.choose the next hop based on 
an SPT. However, a Jobp free path can be guaranteed 
even if it does not represent the shortest distance to a 
destination. For example, any path to a destination that 
traverses, a particular next hop from a router 'S is loop 
free if the distance to. the. destination. continues to de- 
crease at each next hop. 

[001 5] , In one preferred embodiment of the! present in- 
vention viable next. hops jrom a router S, i.e., next hops 
which lie, on a loop free, path to a destination, can be 
guaranteed where the shortest distance from S to the 
destination traversing a given next hop,^minus the cost 
of the link (i.e., its 'distance) from S to that next hop is 
Jess than the absolute shortest distance from S \o the 
destination. For example, if router R is the next hop from 
router 6 along a shortest path from S to destination rout- 
er Di then the (distance between R and D, i.e., d(R, D), 



must be less than the distance from S to D, Le.,d(S, D). 
Extrapolating R to any next router Rj. which jean even 
be a router not on the absolute shortest path between 
S and D, a loop free path from S to D is guaranteed with 

s a next.hop to Rj it d(Rj, D) < d(S^ D). ' 

[0016] Referring to FIG. 4 an example of a network of 
routers is shown, including source router S, destination 
router D and a plurality of routers R-, Each link between 
the routers is numbered and it is assumed that each link 

10 has the same cost, or distance. In this example, it is clear 
that there £re three shortest paths: (i) 1-2-3; (ii) 7-9-1 0; 

and 0J<) 3 ! [1, I '.V .' . "1 , , 
[0017] However, in addition to these shortest paths', 
there are at least twenty four loop free paths: Iri partic- 

is ular, 1-5-6-14-3; 1-5-6-15-16-3; 1-5-6-15-12-13; 
1 -5-6-8-9-10; 1 -5-6-8-9-11 -1 6-3; 1 -5:6-8-9-1 1 -1 2-1 3; 
4-5-2-3; 4-6-15-16-3; 4-6-15-12-13; 4-6-15-11-10; 
4-6-8-9- 10; 4-6-B-9-1 1 -1 6-3; 4-6-8-9-11 -1 2-1 3; 

^ 719-11-16-3; 7-9-'l 1-12-1 3; 7-8-15-16-3; 7-8-15-11-10; 

'20 7-8-15-12-13^ , 7-8-6-14-3; 7-8^6-14-1(5-12-13; 
7^-6-14-1 6-11 -10; 7-8-6-5-2-3; ; 7^-6-5-2 : 1 6-11 -10; 
7t8^6-5:2-16:12-13: A, nbn-viable' loop ► path, however, 
exists for example on ^path 7-8^-5-2-16-1 5. since link 
1 '5. will return the packet to a router previously visited 

25 between links" 8 [ and 

[0018] In the preferred embodiment of the s invem 
the MPA of the present invention makes use of the data 
structure of FIG 1 to compute alternate viable next hops 
from a router whose shortest distances to the destina- 

30 tion are less than the shortest distance from the router 
to the destination. . ' 

[0019] During execution; a conventional SPT algo- 
rithm usually keeps track of certain state information 
about each node in the network. See E : . Dijkstra, "A Note 
35 tb Two Problem^ in Connection with Graphs" Klumerical 
Mamematicsfybl. 1 p.. 115-126 (1959); R. Bellman, "On 
a Routing Proolem\duarteriy of ^plied Mathematics, 
vol. 16, p. B7-9Q (1958), hereby incorporated by refer- 
ence as if f uily set forth herein? Typically, such informa- 
40 tion .consists of a distance attribute 210, the next hop 
attribute from the router 220/ and I perhaps a parent at- 
tribute for each node 230, as shown in FIG. 2. At each 
hop, during the execution i of the SPT algorithm in router 
S.'the distance attribute 2*10 of destination node X rep- 
is resents the distance of the shortest path found so far 
1 from the router S to node X, while the next hop attribute 
r and the parent attribute respectively identify the first hop 
'from router S and the last hop to router S: ' : 
[0020] The data structure of FIG. 1 , however, contains 
so mora state information about each node in the network, 
without incurring significant computational complexity. 
More specifically, a given router S will maintain for each 
destination node X the shortest distance d(S, Xj'frbm 
; .the router S to X. this information can be computed wjth 
ss existing SPT algorithms such as those taught in Dijkstra 
and Bellman, supra: In addition, the distance, 'or cost w 
™' (S,p) for each outgoing link connecting the router S to a 
potential next hop p is also maintained. Furthermore, the 
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data structure keeps track of the length of the shortest 
path that uses p as the next hop from S, which is denoted 
by d p (S, X) When the MPA execution terminates, d p (S, 
X) should equal the sum of w(S f P) and the shortest dis- 
tance found, d(p, X) from p to X By maintaining this data 
structure, the MPA identifies viable next hops for desti- 
nation X using the principles discussed above: namely 
a viable path using next hop p exists if d p (S, X) - w(S, 
»<d(s; r X): ; > - : ■. <■ .= 
[0021] While air next" hops satisfying the above ine- 
quality are viable arid can be revealed by the MPA, it is 
possible for additional viable next hops to exist which 
will not be revealed. By searching through paths, the da- 
ta structure at every node can be updated, increasing 
the^number of viable next hops. As long as the shortest 
distance to every node is computed correctly, the result- 
ing viable next hops will not lead to a bop. 
[0022] With the alternative viable paths for a given 
destination, alternate paths can be constructed by com- 
paring the MPA data structures associated with different 
destinations. Most SPT algorithms known in the art con- 
stantly compare the distance attributes of two neighbor- 
ing nodes The distance attribute of a node eventually 
converges to the shortest distance from the source to 
that node. The distance attribute of node X from source 
S computed by the SPT algorithm is contained in the 
corresponding field 210 of the shortest distance from S 
to X and is updated by the SPT algorithm, independently 
of other operations of the MPA. 
[0023] Referring to FIG. 3, the extended routing algo- 
rithm 300 in accordance with the present invention com- 
prises these modules: MPA Algorithm 301 , Convention- 
al Routing Algorithm 302 and Routing Protocol Memory 
303. The MPA and Conventional Algorithms 301 and 
302, respectively, while possible to be implemented with 
hardware, typically comprise software executed on a 
standard microprocessor having the computing power 
of at least a Power PC or Pentium grade chip. Routing 
Protocol Memory 303 typically comprises at least 4 
megabytes ("MB") of RAM, with 3MB typically allocated 
to the Conventional Algorithm 302 and 1 MB allocated 
to the MPA Algorithm. Furthermore, it is possible to pro- 
vide physically separate RAM chips for use by each of 
the algorithms 301 and 302. Forwarding Table 350 also 
typically comprises at least 4MB of RAM. 
[0024] Every time the SPT algorithm, represented by 
block 302, makes a comparison between nodes a and 
b, the distance attribute from source S to b, d opt (b), is 
equal to the smaller values between its old value and 
the sum of d^a) +w(a, b). 

[0025] Furthermore, with each comparison by the 
SPT algorithm 302 between nodes a and b, MPA block 
301 is triggered and starts its own comparisons. For 
every next hop p from S, the distance from S to b is set 
to the minimum of its old value and d^(S, a) +w(a, b). 
With the termination of the SPT algorithm, the MPA also 
terminates and the data structure is updated in routing 
protocol memory 303. 



[0026] . With the availability of viable next hops the 
. router can select the actual next hop for forwarding a 
packet based on a variety of criteria such as round robin, 
network load, or shortest distance. Those skilled in the 
& ^art will be able to devise various selection criteria. 
■ [0027] Indeed, one such criteria can be used to deter- 
mine satisfactory approximate solutions to.QoS routing 
by decoupling the underlying optimization problem dis- 
cussed above, :into local and global computations. First, 
io multiple viable paths that satisfy the global or less fre- 
quently changed QqS constraints are computed. Then 
among these viable paths, one that is optimized with re- 
spect to the local or more frequently changed QoS pa- 
rameters can be determined. 
*5 [0028] With respect to load balancing it should be not- 
ed that implementation of such a scheme may require 
a means to preserve the FIFO arrival of packets where 
FIFO is assumed by the upper layer protocol at the des- 
; tination router. It may also be necessary to provide ad- 
20 drtional buffer means and processing capabilities at the 
destination router, especially where FIFO is not strictly 
adhered to. o ;/ i : r; y •. 

. [0029] In addition each router can compute and store 
. v in a forwarding table such as 350 inFJG. 3, each viable 1 , 
25 -. next hop for a given destination^ ormaintain only a single 
v viable next hop in forwarding table. 350 while maintain- 
ing additional next hop information; in the routing proto- 
col memory 303. 
. [0030] By maintaining, multiple viable next hops in a 
30 router for; each destination, recovery from linkfailure is 
- much faster than heretofore possible, since the recovery 
> mechanism using alternate paths can be implemented 
locally by the: router, and there is almost no delay in- 
curred by the link failure. On the other hand, without 
3S , MPA, after detecting a:link failure, the router would need 
to broadcast throughout the networkjhe linkfailure, then 
recompute the shortest path tree to determine a new al- 
ternative, path. ! i .=. = ; ; , 
[0031] A more exhaustive search of , alternate next 
40 hops can be obtained by running a set of independent 
SPT algorithms in parallel: For every next hop pj any 
r SPT algorithm including adynamic algorithm, i.e., an al- 
gorithm which can compute a new SPT by readjusting 
the old SPT, is executed using only.the distance attribute 
45 : d p (S,X) in the data structure associated with.the desti- 
nation X. (While a static algorithm recomputes an entire 
shortest path, a dynamic algorithm recognizes that the 
majority of the path segments will remain the same and 
' therefore only certain segments are. recomputed). After 
so , the SPT algorithm is executed,. the data structure is up- 
dated and all otthe next hops pjithat, satisfy the above 
inequality are obtained. -o . 



55 Claims 

1 . A method for routing a data packet from a router to 
a destination node in a packet switching network, 
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3. 



said data packet originally from a source node in 
• said packet switching network, said method com- 
prising the steps of: - ■ - - • ■ t - . , 

* determining one or more viable next hops from 
said router; each of said viable next hops tying 
* ■ 'on a path from said source node to said desti- 
nation node, said path having a distance which 
decreases at each next hop along said path un- 
" til said destination node; .« / v K 

storing in a first memory means at said router 
said one or more viable next hops; and 
selecting a first one of said one or more viable 
next hops to forward said data packet.' 

A method according to Claim 1 wherein each of said 
viable- next hops are in a path from said destination 
whose shortest distance is less than the absolute 
shortest distance from said source node to said 
destination node. 

A method according to Claim 1 further comprising 
the step of selecting a second oner of 6aid one; or 
more viable next hops if it is determined that a link 
failure exists along said path of said first viable next 
hops and a second viable next hop is stored in said 
first memory means: 
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the shortest distance from said router to a des- 
x tination node; 

the distance to each next hop from said router; 
,v- and" ■ \- ■- , ....... 

-i foreach of said next hops, the distance of the 
v,< shortest path from said router to said destina- 

- - tion node using said next hop; * » \ 

\ computer implemented means t or determining 
one or more viable next hops from said data 
? ; structure: each of said -viable next hops satis- 
: \* > fying the inequality that the difference between 
. jrV4he distance from said routerto said destination 
;;' node traversing one 9f said viable next hops, 
r and the cost from said router to said next hop 
r i. = is less than the shortest distance from said rout- 

- erto said destination node., v * . . - 

C>. *.»! . - f " : "■ • " - • 
>A?data structure for . use in. a router, in a packet 
i Switching network, comprising; n . ; •. ;. 
t- ■ ft - r- ' -i . : " " "\ . v : . 1 ' y 
i . .-. the shortest distance from said router, to a des- 
A tination node; ; ■. ; , .. 1 * ..v <w, : 
»-.=■ the cost of each. next hop from said router; and 

sj. :for each of said next hops, the distance; of the 
; - ^shortest path from said router to said destina- 

; :tion node using said next hop. ^ 



A method according to Claim 1 wherein each of said 
"viable next hops leads to a loop free path from said: 
source node to said destination node, but is not con- 
strained to lead to a shortest path between- said 
source node and said destination node,; 



30-r;, 



A method according to Claim 1 wherein said select-^ 
ing steps are constrained to select a viable next hop 
leading to the shortest path as compared with all of 
said one or more viable next hops stored in said first 
memory means.-; ; ^ ; 



35 r 



6. A method for. determining multiple loop free paths 
from a source nodetpadestinatioanode in a packet 
switching network, said packet switching network 
comprising a plurality of routers;csaid method com- 
prising the step of selecting at each of said routers, 
one or more viable next hops for forwarding a data 
packet originating from a source node, t to a destina- 
tion mode, each rof said viable next hops con- 
strain edby the inequality that the distance from said 
next hop Xo said destination node is, less than the 
shortest distance from said source node to said 
destination node. o J . ; - •. ..*> , 
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7. A router for use in a packet switching network for 
routing a data packet originating from a source 
node, to a destination node, comprising: 



ss 



m a data structure comprising,; 
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